ED 292 680 



DOCUMENT RESUME 



SE 049 061 



AUTHOR 
TITLE 



SPONS AGENCY 
PUB DATE 
GRANT 
NOTE 



PUB TYPE 



Grouws, Douglas A,; Good, Thomas 

Teaching Mathematical Problem Solving: Consistency 

and Variation in Student Performance in the Classes 

of Junior High School Teachers • 

National Science Foundation, Washington, D.C. 

Apr 88 

MDR-8470265 

19p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (New 
Orleans, LA, April 5-9, 1988). 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MFOl/PCOl Plus Postage. 

Junior High Schools; Junior High School Students; 
*Mathematics Achievement; Mathematics Curriculxun; 
Mathematics Education; *Mathematics Instruction; 
*Mathematics Teachers; *Problem Solving; Secondary 
School TeachersL-; Teacher Behavior; *Teacher 
Effectiveness 

Mathematics Education Research 



ABSTRACT 

This study examined student growth in mathematical 
problem solving and relationships between teachers and changes in 
class problem-solving performance over a school year. The sample for 
the study involved 24 junior high school mathematics teachers, 119 
mathematics classes, and more than 2,500 students. Classes were 
tested using a 10 item problem-solving test at the beginning of the 
school year and at the end of the school year and increases in 
problem-solving performance were computed. Classes differed 
substantially in the amount of increases in performance. The five 
teachers with the largest class mean gains (highs) were compared with 
the five teachers with the lowest mean gains (lows). The high 
teachers taught 22 classes and 100 percent of these classes gained 
more than was statistically predicted from pretest scores. For the 
low teachers, 75 parcent of their classes gained less than was 
predicted. Thus, the teacher, and indirectly teaching behavior, seems 
to be an important factor in increasing student performance in the 
domain of mathematical problem-solving. .(Author/PK) 
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Teaching Mathematical Problem Solving • 1 
Abstract 

This study examined student growth in mathematical problem solving and 
relationships between teachers and changes in class problem-solving 
performance over a school year. The sample for the study involved 24 junior 
high school mathematics teachers, 119 mathematics classes, and more than 2500 
students. Classes were tested using a 10 item problem-solving test at the 
beginning of the school year and at the end of the school year and increases 
in problem-solving performance were computed. Classes differed substantially 
in the size of their increases in performance indicating that much more 
problem-solving learning takes place in some mathemacics classrooms than in 
others. Teachers were associated with both the size and consistency of these 
gains. The 5 teachers with the largest class mean gains (Highs) were compared 
with the 5 teachers with the lowest mean gains (Lows). The high teachers 
taught 22 classes and 100 percent of these classes gained more than was 
statistically predicted from pretest scores. For the low teachers, 75 percent 
of their classes gained less than was predicted. Thus, teacher, and 
indirectly teaching behavior, seems to be a powerful factor in increasing 
student performance in the domain of mathematical problem solving. 
Naturalistic study of teaching practice and teacher beliefs of high performing 
teachers is an important direction for future research. 
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Teaching Mathematical Problem Solving; Consistency and Variation 
in Student Performance in the Classes of Junior High School Teachers 

Douglas A. Grouws 
Thomas L. Good 
University of Missouri-Columbia 
One would be hard pressed to find a period in the history of mathematics 
education where attention to problem solving in some form was not vigorously 
advocated for alK Dower (1928) indicated that the "mastery of the 
fundamental facts and processes is not the ultimate end of arithmetical 
instruction," but rather life demands that students have the ability to 
interprets comprehend, and solve quantitative problems that arise in everyday 
situations. According to Reeve (1940), even a reduced program of mathematical 
study should emphasize problem solving and modes of thinking. Similar 
reconmiendations are found in more recent literature. 

In its Agenda for Action (1980), the National Council of Teachers of 
Mathematics (1980) recommends that problem solving be the focus of all school 
mathematics • This position recognizes that the value of mathematics accrues 
from being able to apply it to problem-solving situations. Begle (1979) 
points out that "The real justification for teaching mathematics is that it is 
a useful subject and, in particular, that it helps in solving many kinds of 
problems*" 

Considering the long history of recommendations for improved classroom 
problem-solving instruction , it is surprising that most research has focused 
on individual students, usually in laboratory settings, rather than on actual 
classroom teaching and learning. Grouws (1985) points out that there is a 
dearth of studies of the teacher* s role in and contribution to student 
learning of problem solving. Furthermore, there appears to be a lack of 
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Teaching Mathematical Problem Solving • 3 

financial investment in instructional research on various mathematical topics 
(Good & Bicidle, 1988) • 

Another reason for the paucity of classroom research in this area may be 
that a variety of perspectives on how problem solving should be taught have 
been advocated in recent years. These can be roughly classified into five 
categories (Kilpatrick, 1985): osmosis, memorization, irritation, cooperation, 
and reflection. Most of these recommendations, however, stem from logical 
analysis, personal experience, or current ideas about student learning rather 
than from systematic study of classroom practice. However, there have been 
some successful attempts to improve problem solving instruction (e.g., Charles 
& Lester, 1984). 

Perhaps another rc^ason for the lack of naturalistic classroom problem 
solving research is a pessimistic view that suggests that problem solving is 
rarely taught, that such instruction is of poor quality, and that students' 
problem-solving performance is dismal (Lester, 1985). Indeed there is 
empirical evidence that problem-solving instruction in some classrooms is 
relatively uninspiring (Burns & Lash, 1984). Based on case studies. Stake and 
Easley (1978) strongly suggest that the classroom environment may have to be 
drastically changed if we wish students to develop logical thinking skills and 
improved problem solving ability. 

Student performance data also indicate that there is a need for 
substantial improvement in how well students solve problems. Data from the 
Third National Assessment of Educational Progress (Carpenter, Matthews, 
Lindquist, & Silver, 1984) show that the majority of students at all age 
levels have difficulty with any nonroutine problem that requires some analysis 
or thinking. One conclusion from the Second International Study of 
Mathematics Achievement (McKnight et al., 1987) is that U.S. students are well 
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below the international average in such areas as problem solving. In summary, 
there are good reasons not to be satisfied with problem-solving instruction in 
many classrooms and to be disappointed with students' problem-solving 
performance in general. 

The preceding notwithstanding, it may be that this bleak picture of 
mathematics instruction is too generalized. That is, there certainly are some 
students who are becoming excellent problem solvers. For example, selected, 
talented U.S. students did well in the recent international competition 
sponsored by the International Commission on Mathematics. There prob?.bly are 
also entire classrooms in which most students are making substantial progress 
in developing their problem-solving ability. Of the more than 250,000 
mathematics teachers in the U*S., there likely are some who foster large 
increases in student problem-sol ing ability in every class they teach. Due 
to an increased emphasis on problem solving, some mathematics education 
programs and school districts are probably relatively successful in helping 
classroom teachers to become more effective problem-solving teachers. There 
is a need for objective evidence on this latter point, however, and careful 
study of the classrooms and students involved. Particularly needed is 
attention to the quality of instruction. Recent research (e.g.. Good & 
Grouws, 1987) suggests that under certain conditions, assisting teachers to 
attend to the quality of the development component of their lessons can 
positively influence student performance in areas requiring higher-order 
thinking. 

The purpose of this paper is to discuss data from the first stage of a 
research project that examines a large sample of teachers and the effects of 
their instruction on the problem solving ability of their students across an 
entire school year. We wanted to see if v^jriation in students* learning of 
problem-solving skills was consistently associated with classroom instruction. 
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Teaching Mathematical Problem Solving - 5 
The Study 

To study problem-solving performance one must first carefully define 
problem solving. Silver and Thompson (1984) point out that readers of the 
problem-solving literature are confronted with a wide variety of definitions 
of problem solving. There is agreement, however, that what constitutes a 
problem is idiosyncratic; that is, what represents a problem for one student 
may not be a problem for another student. There also is a consensus that a 
problem involves a situation where something is to be found or shown and the 
way to do this is not immediately obvious to the solver. However, as Shulman 
(1985) has suggested, when analyzing problem solving it may be beneficial to 
specify some of the details concerning the kind of problem solving under 
analysis rather than argue about whether something is or is not "real" problem 
solving. 

Problem Solving Measure 

In spite of the importance ascribed to problem solving over the years and 
the common elements in most definitions, there is little agreement on how to 
measure problem-solving ability (Schoen & Oehmke, 1980). Charles, Lester, and 
O'Daffer (1987) have suggested alternative means of assessing problem solving 
and correctly emphasized that the evaluation procedures must take into account 
the purpose of the assessment. 

The problem solving assessment in the current study was for screening 
purposes. A ten-item paper and pencil instrument composed of a variety of 
situations of varying difficulty that would constitute problems for most 
students in the sample was developed. The test was designed to involve more 
than simple word problems that would only be routine exercises for students, 
yet not be so challenging that it would be beyond the capacity of most 
seventh- and eighth-grade students. Table 1 displays for each itam a 
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Insert Table 1 about here 



description, point bi serial coefficient, item difficulty, and standard 

deviation based on all students taking the pretest administration of the 

instrument (N = 2540). The test items were wr-^^-ten to meet given descriptive 

criteria (see Table 1) and to span a continuum from relatively simple verbal 

problems to nonroutine problems. The following problem from the instrument (# 

9) reflects a problem of the nonroutine type. 

How many times must the number keys on a typewriter be hit in order 
to type page numbers on a paper that has 124 pages? 

None of the problems involved just two numbers that had to be simply combined 

with an arithmetic operation to obtain a correct solution. A mathematics 

educator outside the project staff classified items by descriptive category, 

and these data confirmed our original judgments. The KR-20 coefficient of 

internal consistency for the instrument was .73 (N = 2540). 

Sample 

The sample was composed of 24 teachers who taught mathematics in one of 
the six junior high schools in a large midwestern school district. This 
volunteer sample represented about 80 percent of the junior high mathematics 
teachers in the district. These teachers taught 119 classes composed of more 
than 2500 students. All students at grade 7 and grade 8 in the school 
district took one of the following courses each year: Math 7, Advanced Math 
7, Math 8, PreAlgebra, or Algebra. 
Pr ocedure 

Teachers administered the problem-solving instrument to each of their 
seventh- and eight-grade mathematics classes during tlie same three day period 
in early October. Teachers used st?<ndardi;!ed directions to give the test and 
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Teaching Mathematical Problem Solving - 7 

students had the entire class period to complete the test. During the school 

year each teacher was observed on a regular basis. Teachers administered the 

problem solving instrument to each class again in early May during a two day 

period. 

Data Analysis 

For each class that had a complete data set pretest and posttest problem 
sol zing mean scores were computed. Linear regression on these pairs of means 
was used to generate a linear equation that predicted class posttest mean 
scores based on class pretest mean scores. A residual class mean (posttest 
score minus predicted score) that indicated gain beyond that predicted was 
then computed for each class. 

Results 

Student Performance 

The mean student score on the problem solving pretest was 4.8 problems 
correct (s.d. = 2.3); the posttest mean was 5.8 problems correct (s.d. = 2.3) 
Thus, on the 10-problem test, students scored an average of one more problem 
correct on the posttest than they did on the pretest. Table 2 shows the 
pretest and posttest means for each test item. The greatest growth in 
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performance Wv'^s on items 2 and 7, and the smallest increase was on item y. 
Item 2 was a two-step problem that required students to subtract two numbers 
and then subtract that result from a third number. Item 7 was classified as ^ 
diagram-type problem because comprehension of the problem was greatly 
facilitated by making a drawing or sketch. The problem involving the least 
improvement was a nonroutine problem (#9) in the sense that it only included 
one number so immediately performing an arithmetic operation was not possible 
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There were substantial differences in performance gains among classes. 
The smallest pretest to posttest gain for a class was -1.06, and the class 
with the largest gain had an increase of 2.92. The residualized mean scores 
for these classes were -2.01 and 1.95, respectively. 
Teacher Data 

Class residual means were aggregated by teacher and a mean of these means 
was completed. Table 3 displays these means by teacher in rank order from 



Insert Table 3 about here 



largest to smallest. The largest teacher mean was .80, which implies that on 
average each of these teacher's five classes gained .80 mc.e than was 
predicted. For example, in one of these classes the :nean pretest score was 
5.46 and the associated predicted posttest score was 6.38. The actual mean 
posttest score was 7.21, giving a residualized mean of .83 for this class. 

As an indication of the stability of teacher means, a proportion was 
examined. For each teacher, the number of classes with more than expected 
gains was compared to the total number of classes (see Table 3). There were 
22 classes taught by the five highest teachers. In 100 percent of these 
classes the class mean exceeded the posttest mean predicted by the pretest 
mean scores. In contrast, there were 20 classes taught by the five lowest 
teachers. In 15 of these classes, or 75 percent, the class mean did not reach 
the level predicted by the pretest results. 

Discussion 

In considering the implications of the data from this study it is 
important to remember that it was a naturalistic study and thus a number of 
factors could not be controlled. For example, not every class in the sample 
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used the same textbook, but we did insure that every type of class (e,g,. Math 
7, Math 8, and so on) did use the same textbook. Not every teacher taught the 
exact same mix of classes but we did examine the sample with this in mind and 
found that generally teachers taught a variety of classes (e.g,, no teacher 
taught just prealgebra classes). Further,, not all classes had students of 
exactly the same ability. Adjusted posttest scores were used to attempt to 
statistically account for such differences. In summary, we were sensitive to 
the potential importance of such factors and found no systematic patterns that 
would seem to bias the findings we have reported. The relatively large sample 
used in the study should also tend to diminish the importance of some factors 
that might otherwise have been of concern. Further, in consideration of the 
nature of the study we have also focused on reporting patterns of results 
rather than the results from a single teacher or class. 

With the preceding qualifications in mind, the data from this study lend 
support to several generalizations. There is important variation in teachers ^ 
effects on student problem solving performance in junior high school 
mathematics classes. In particular, within extant practice there are teachers 
who in a given school year obtain substantially better problem-solving results 
than would be predicted. It is important that these favorable outcomes 
occurred in every mathematics class the teacher had responsibility for that 
school year. This argues against the effects being random in nature or 
attributable to a particular type of class or a particularly suitable 
combination of teacher and class type. Thus, the data support the contention 
that some teachers offer effective problem solving instruction and that what 
these teachers do in instruction merits research. 

It is important to emphasize strongly at this point that we do not 
maintain that the instruction taking place in the aforementioned classrooms is 
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good or bad nor that other teachers should emulate it. It is possible that 
examination of the problem solving gains in greater depth through other 
procedures (e^g..^ observation of individuals solvino problems, with more 
attention given to thinking strategies employed and techniques used) will 
temper the results. It may be that when problem solving is defined 
differently, the gains may not be as impressive. Similarly it is possible, 
though r.x/t very likely in our view, that the results were obtained using 
inappropriate instructional methods (such as a focus on rote practice) or at 
the expense of other important outcomes (such as student interest or 
attitude). It is important, however^ that we now have data to begin to 
describe the results of contemporary practice. These data are encouraging in 
that they suggest that careful study of extant practice in mathematical 
problem solving has potential for making a contribution to our understanding 
of how student problem solving ability may be improved in classroom settings. 
Again, we stress that an examination existing practice is but one basis for 
developing an understanding of problem solving instruction and learning. 
Empirical studies of student learning and new models and theories of problem 
solving learning that go beyond extant practice are al.^o m^sded. Still we 
believ,3 that an understanding of current practice — the good, the typical, 
and the problematic — is an important part of the dialogue. 

Future Research 

We have previously mentioned that there is a lack of agreement as to how 
problem solving should be measured. To conduct the present research we 
operationally defined problem solving and developed an instrument that 
reflected a conceptualization of problem solving that was responsive to 
practicing teachers • views as well as professional recommendations. An 
important question to investigate is whether the same teachers are effective 
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if different student measures are used. A prediction based on a theoretical 
position should be made, instruments should be developed, and data to help 
answer the question gathered. Research of this type is essential in answering 
the many arguments that exist on both sides of the issue. 

Research should also examine the effects of these teachers' beliefs and 
attitudes on their problem-solving instruction. Do these successful teachers 
have a greater interest in problem solving than do other teachers? McLeod 
(1987) suggests that affective considerations in problem solving at both the 
student and teacher level have not been studied adequately. How teachers view 
student affective outcomes may also be important. Similarly, a teacher's 
belief structure may influence what content is emphasized, how it is 
presented, and in turn, what the students eventually learn (Cooney, 1985). 
These issues are important in developing a more informed view of the 
relationship between the teaching of problem solving and problem solving 
learning. 
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Table 1 

Problem Solving Test Items by Type, Point Biserial Correlation, 
Item Difficulty, and Standard Deviation 



Problem 


Problem 
lype 


Point 
D iseri a i 


Item 
Ul TTICU ity 


Standard 
Deviation 


1 


Extra Information 


.48 


.82 


.39 


2 


Two-Step Problem 


.55 


.41 


.49 


3 


Extra Information 


.62 


.23 


.42 


4 


Prnnnrfinn Prnhl pm 


. 00 


• 0^ 




5 


Implied Number 


.46 


.82 


.39 


6 


Work Backward 


.59 


.43 


.50 


7 


Diagram 


.49 


.72 


.45 


8 


Guess and Check 


.62 


.29 


.45 


9 


Nonroutine 


.40 


.06 


.24 


10 


Nonroutine 


.5? 


.67 


.47 
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Table 2 

Differences in Performance on Problems from 
the Pretest to the Posttest 



Problem Number 


Pretest Mean 


Posttest Mean 


U 1 1 1 C( Cllti#C 


1 


.82 


.88 


.06 


2 


.41 


.56 


.15 


3 


.23 


.33 


.10 


4 


.39 


.51 


.12 


5 


.82 


.87 


.05 


6 


.43 


.56 


.13 


7 


.72 


.78 


.06 


8 


.29 


.44 


.15 


9 


.06 


.10 


.04 


10 


.67 


.78 


.11 
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Table 3 

Ordered List of Mean Residual Problem Solving Gain Scores 



Teacher 
Number 


Number of 
Classes 


Number of Classes 
Pos. Res. Gains 


Mean of 
Residual Gains 


13 


5 


5 of 5 




15 


4 


4 of 4 




9 


5 


5 of 5 


44 


19 


3 


3 of 3 




21 


5 


5 of 5 


• oo 


6 


6 


2 of 6 




4 


5 


2 of 6 


07 


7 


3 


2 of 3 


07 


12 


6 


4 of 6 


OR 


11 


5 


2 of 5 


02 


23 


5 


1 of 5 


- 01 


8 


6 


4 of 6 


- 02 


14 


6 


5 of 6 


- 03 


3 


5 


1 of 5 


- 11 


25 


5 


2 of 5 


- 13 


2 


6 


2 of 6 


- 14 

• x^ 


17 


3 


0 of 3 


- 20 


16 


6 


2 of 6 


- 23 


22 


4 


2 of 4 


- 24 


24 


6 


2 of 6 


-.30 


18 


1 


0 of 1 


-.32 


10. 


5 


1 of 5 


-.43 


c 


3 


1 of 3 


-.44 


1 


5 


1 of 5 


-.58 




^The 24 teachers were assigned numbers using the numbers 1 to 25; no 
teacher was assigned number 20 in this process* 
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